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I. Introduction 

The best codes of short or intermediate blocklength that 
have so far been discovered are usually defined in ways that 
do not immediately suggest a sparse parity check matrix rep- 
resentation. For this reason, large classes of classical text- 
book codes, which would give excellent performance under 
optimal decoding, have been mostly ignored as candidates for 
the belief propagation (BP) decoding algorithm. A notable 
exception are the one-step majority logic decodable codes, 
which have been shown to have excellent error-correcting per- 
formance when decoded using BP [1]. We report here on a 
method for generating sparse generalized parity check (GPC) 
matrix representations of other classical codes. 

II. Improving GPC Matrices 

An M by N generalized parity check (GPC) matrix repre- 
sentation [2] of an (n,&) binary linear block code will have 
N > n columns, and M > N — k rows, where the M rows 
span an N — k dimensional sub-space. 

The BP algorithm seems to work best on GPC matrices 
that have the following characteristics: 

1. The number of ones in each row is small. 

2. The number of ones in each column is large. 

3. For all pairs of rows of the matrix, the number of 
columns that have a one in both rows is small; ideally 
zero or one. 

We developed an algorithm that improves GPC matrices in 
all three of these characteristics; but the new GPC matrices 
also have additional auxiliary bits, and because there is no 
evidence from the channel to determine the value of these 
bits, they may cause performance to deteriorate on channels 
other than the binary erasure channel (BEC). 

Given an M by N input GPC matrix H, our algorithm 
outputs an M f by N* GPC matrix H* . The basic idea behind 
the algorithm, which is described in detail elsewhere [3], is to 
re- write constraints involving large numbers of bits by using 
auxiliary bits that encode the parity of sets of bits. Using 
this "divide-and-conquer" approach, we try to minimize the 
number of bits involved in each constraint by re-writing the 
constraints in terms of sets of bits that are as large as possible. 
We also try to use as many redundant constraints as possible. 

III. Results for a Euclidean Geometry Code 

We present some empirical results obtained applying our 
method on a (n = 255, k = 127) Euclidean geometry (EG) 
code. In the notation of [4], the code we studied has param- 
eters, m = 4, s = 2, and /i = 1. In the geometric interpre- 
tation of this code, each bit corresponds to a point, there are 
21 * 255 = 5355 lines each consisting of four points, and there 
are also 5355 planes consisting of four parallel lines. This code 
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can be represented by a redundant parity check matrix H with 
M = 5355 and N = 255, where each row has weight 16. 

Applying our algorithm to this input GPC matrix H, we 
obtain an output GPC matrix H* with M' = 32130 and 
N* = 5610. The weights of the rows are either four or five. 
Results using BP decoding with these matrices on the BEC 
are shown in Figure 1, where we have also shown the per- 
formance of BP decoding of regular (3, 6) Gallager codes and 
maximum likelihood (ML) decoding of random linear codes 
with parameters (n = 256, k - 128) [5]. 

(255.127) EG code on binary eraaure channel 
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Figure 1: Performance of EG (255, 127) code on the BEC. 

The results using BP decoding with H are already quite 
good-much better than BP decoding of Gallager codes. This 
is a consequence of using highly redundant parity check matri- 
ces with 5355 rows instead of just 128 rows. The performance 
using H f was even better, though still clearly distinguishable 
from ML decoding. 
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Abstract 

The best codes of short or intermediate blocklength that have so far been discovered 
are usually defined in ways that do not immediately suggest a sparse parity check ma- 
v trix representation. For this reason, large classes of classical textbook codes, which 
would give excellent performance under optimal decoding, have been mostly ignored 
as candidates for the belief propagation (BP) decoding algorithm. A notable exception 
are the one-step majority logic decodable codes, which have been shown to have excel- 
lent error-correcting performance when decoded using BR We report here on a method 
for generating sparse generalized parity check (GPC) matrix representations of other 
classical codes. 
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Abstract 

Important inference problems in statistical physics, computer vision, error-correcting 
coding theory, and artificial intelligence can all be reformulated as the computation of 
marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an 
efficient way to solve these problems that is exact when the factor graph is a tree, but 
only approximate when the factor graph has cycles. 

We show that BP fixed points correspond to the stationary points of the Bethe ap- 
proximation to the free energy for a factor graph. We explain how to obtain region- 
based free energy approximations that improve the Bethe approximation, and corre- 
sponding generalized belief propagation (GBP) algorithms. 

* We emphasize the conditions a free energy approximation must satisfy in order to be 
a "valid" approximation. We describe the relationship between four different methods 
that can be used to generate valid approximations: the "Bethe method," the "junction 
graph method," the "cluster variation method," and the "region graph method." 

The region graph method is the most general of these methods, and it subsumes all 
the other methods. Region graphs also provide the natural graphical setting for GBP 
algorithms. We explain how to obtain three different versions of GBP algorithms and 
show that their fixed points will always correspond to stationary points of the region 
graph approximation to the free energy. We also show that the region graph approxi- 
mation is exact when the region graph has no cycles. 
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